Realtime control of sequence generation with character based Long Short Term Memory Recurrent Neural Networks

Recurrent Neural Networks (RNNs) — particularly Long Short Term Memory (LSTM) RNNs — are a popular and very successful model for generating sequences. However, most LSTM based sequence generation techniques are currently not interactive and do not allow continuous control of the sequence generation, let alone in a gestural or expressive manner. This research investigates methods of realtime continuous control and steering of RNN sequence generation, as well as ways of expressively controlling the output.